Implements Task 11.3.1 - Analysis and Balance Reporting. Creates tooling to consume aggregated sweep data and generate actionable balance reports with statistical rigor.

Changes

New CLI Tool: scripts/analyze_balance.py

• 4 subcommands: report, regression, trends, stats
• Statistical analysis: confidence intervals (t-distribution), two-sample t-tests, linear regression trend detection
• Balance detection: dominant strategies (>10% win rate delta), underperforming mechanics (<5% usage), unused story seeds, parameter sensitivity by difficulty
• Visualizations: win rate bar charts, trend line charts, action distribution pie charts (matplotlib with base64 embedding)
• Output formats: Markdown, HTML (with embedded charts), JSON

Tests: tests/scripts/test_analyze_balance.py

• 39 tests across 12 test classes (requirement was 12+)
• Covers statistical calculations, report generation, CLI commands, edge cases (empty DB, single run)

Usage

# Generate Markdown balance report
uv run python scripts/analyze_balance.py report --database build/sweep_results.db

# Generate HTML report with charts
uv run python scripts/analyze_balance.py report -d build/sweep_results.db -f html -o report.html

# Detect regressions between runs
uv run python scripts/analyze_balance.py regression -d build/sweep_results.db -b 1 -c 2

# Show metric trends
uv run python scripts/analyze_balance.py trends -d build/sweep_results.db --days 30

# Statistical analysis (CIs and t-tests)
uv run python scripts/analyze_balance.py stats -d build/sweep_results.db --json

Processes SQLite database from aggregate_sweep_results.py (Task 11.2.1). Falls back gracefully when scipy unavailable.

Created from Copilot CLI via the copilot delegate command.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.